feat(security): launch Penetration Tests epic across app, API lifecycle, and provider integration by claudfuen · Pull Request #2193 · trycompai/comp

claudfuen · 2026-02-27T01:55:18Z

Summary

This PR ships the new Security → Penetration Tests capability end-to-end.

It introduces a complete one-time penetration test lifecycle:

create run
mocked checkout handoff/return (temporary)
queue/progress visibility
run detail page
markdown/PDF artifact access

Final architecture

Frontend calls the existing Comp API client.
Nest API exposes dedicated penetration-test endpoints.
Nest API integrates directly with Maced.
No Next.js API proxy routes are used for this feature.

User flow

User opens Security → Penetration Tests.
User submits a target URL (and optional repo URL).
Frontend calls POST /v1/security-penetration-tests.
API creates a provider run at Maced and stores org ownership mapping (organizationId + providerRunId).
User sees the run in list/detail pages with in-progress polling.
On completion, user can access markdown and PDF report artifacts.

Canonical provider contract

Maced canonical run identifier is id (no compatibility fallback path in Comp for runId).
Comp stores ownership against providerRunId mapped from provider id.
Webhook completion/failure payload handling is aligned to provider id.

What changed

Frontend (`apps/app`)

Added Security pages:
- /:orgId/security
- /:orgId/security/penetration-tests
- /:orgId/security/penetration-tests/:reportId
- /:orgId/security/penetration-tests/checkout
Added polished empty/list/detail states for penetration tests.
Added SWR hooks and feature-specific client contracts:
- usePenetrationTests
- usePenetrationTest
- usePenetrationTestProgress
- useCreatePenetrationTest
Added artifact actions on detail page:
- markdown view
- PDF download
Added Security nav/search integration in app shell.
Added PostHog UI/route gating via is-security-enabled.

Backend (`apps/api`)

Added dedicated module:
- SecurityPenetrationTestsModule
Added endpoints:
- GET /v1/security-penetration-tests
- POST /v1/security-penetration-tests
- GET /v1/security-penetration-tests/:id
- GET /v1/security-penetration-tests/:id/progress
- GET /v1/security-penetration-tests/:id/report
- GET /v1/security-penetration-tests/:id/pdf
- POST /v1/security-penetration-tests/webhook
Added Maced client utility with strict Zod validation for provider request/response contracts.
Added strict org ownership enforcement on list/get/progress/artifact APIs.
Added webhook handshake + token verification + idempotency tracking.

Database (`packages/db`)

Added ownership mapping model/table:
- SecurityPenetrationTestRun
- table: security_penetration_test_runs
Ownership mapping key:
- organizationId + providerRunId (unique)

Docs (`packages/docs`)

Added customer-facing Penetration Tests docs page.
Updated OpenAPI docs for penetration-test route and schema names.

Environment variables

Required

Variable	Service	Purpose
`MACED_API_KEY`	`apps/api`	API key for Nest API to call Maced provider endpoints.

Optional

Variable	Service	Default	Purpose
`MACED_API_BASE_URL`	`apps/api`	`https://api.maced.ai`	Override Maced base URL (staging/dev/custom).
`SECURITY_PENETRATION_TESTS_WEBHOOK_URL`	`apps/api`	none	Base callback URL used to build webhook target for new runs.

Notes:

is-security-enabled is a PostHog feature flag (not an env var).
No feature-specific frontend env vars are required.

Rollout

Security tab and Security routes are gated in product UI by PostHog flag:
- is-security-enabled

Out of scope (intentional)

Live Stripe checkout + reconciliation (current flow uses mocked checkout).

Risk/cost note

Each provider generation has meaningful token cost.
Generation remains explicit user-driven one-time submit (no automatic recurring runs).

vercel · 2026-02-27T01:55:21Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
app	Ready	Preview, Comment	Mar 2, 2026 8:09pm
portal	Ready	Preview, Comment	Mar 2, 2026 8:09pm

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eb7a64de99

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

apps/app/src/app/api/security/vulnerability-reports/route.ts

apps/app/src/app/api/security/vulnerability-reports/[id]/report/route.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d2df06e6b9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

apps/app/src/app/api/security/vulnerability-reports/webhook/route.ts

apps/app/src/app/api/security/vulnerability-reports/route.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5e4c528f51

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

apps/app/src/app/api/security/vulnerability-reports/_api-proxy.ts

apps/api/src/security-vulnerability-reports/security-vulnerability-reports.service.ts

… Maced webhook reconciliation

apps/api/src/security-vulnerability-reports/security-vulnerability-reports.service.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6a1e273ab0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

apps/app/src/app/api/security/vulnerability-reports/_api-proxy.ts

apps/api/src/security-penetration-tests/security-penetration-tests.service.ts

…eports-module

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be16674dad

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

apps/api/src/security-vulnerability-reports/security-vulnerability-reports.service.ts

…typing

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 789fb5eb34

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

apps/api/src/security-penetration-tests/maced-client.ts

…ider id contract

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7273e28c53

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

apps/api/src/security-penetration-tests/security-penetration-tests.service.ts

…calls

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 77ab81e683

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

apps/api/src/security-penetration-tests/security-penetration-tests.service.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 49f6031f4d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

packages/db/prisma/migrations/20260302113000_add_security_penetration_test_runs/migration.sql

apps/api/src/security-penetration-tests/security-penetration-tests.service.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b91d508b9e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-02T20:17:30Z

apps/app/src/app/(app)/[orgId]/security/penetration-tests/hooks/use-penetration-tests.ts

+        if (response.status < 200 || response.status >= 300) {
+          throw new Error(response.error ?? `Request failed with status ${response.status}`);


Surface provider error details on create failures

This branch throws response.error, but the browser API client currently only populates that field from data.message (not data.error), while the new penetration-test backend paths commonly return errors as { "error": ... } (for example in security-penetration-tests.service.ts). In those cases users get a generic HTTP 4xx message instead of the real validation/provider error, which makes create-run failures hard to diagnose and breaks the UX expected by this flow.

Useful? React with 👍 / 👎.

claudfuen · 2026-03-02T20:26:13Z

🎉 This PR is included in version 1.87.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

feat(app): add security vulnerability report module

eb7a64d

vercel bot temporarily deployed to Preview – portal February 27, 2026 01:55 Inactive

vercel bot had a problem deploying to Preview – app February 27, 2026 01:56 Failure

fix(app): stabilize shell tooltip and menu ids

fa690f4

vercel bot temporarily deployed to Preview – portal February 27, 2026 01:58 Inactive

chatgpt-codex-connector bot reviewed Feb 27, 2026

View reviewed changes

apps/app/src/app/api/security/vulnerability-reports/route.ts Outdated Show resolved Hide resolved

apps/app/src/app/api/security/vulnerability-reports/[id]/report/route.ts Outdated Show resolved Hide resolved

fix(app): wrap shell tooltips in provider for hydration

d2df06e

vercel bot temporarily deployed to Preview – portal February 27, 2026 01:59 Inactive

vercel bot had a problem deploying to Preview – app February 27, 2026 02:01 Failure

feat(security): add vulnerability reports checkout flow and report pages

3216d59

chatgpt-codex-connector bot reviewed Feb 27, 2026

View reviewed changes

apps/app/src/app/api/security/vulnerability-reports/webhook/route.ts Outdated Show resolved Hide resolved

apps/app/src/app/api/security/vulnerability-reports/route.ts Outdated Show resolved Hide resolved

feat(security): add vulnerability reports module and checkout flow

2f0546f

vercel bot temporarily deployed to Preview – portal February 27, 2026 02:34 Inactive

claudfuen changed the title ~~Add security vulnerability reports feature with API-backed queue~~ feat(security): add security vulnerability reports module and checkout lifecycle Feb 27, 2026

vercel bot had a problem deploying to Preview – app February 27, 2026 02:35 Failure

feat(security): gate security tab and routes behind feature flag

5e4c528

vercel bot temporarily deployed to Preview – portal February 27, 2026 02:35 Inactive

vercel bot had a problem deploying to Preview – app February 27, 2026 02:36 Failure

chatgpt-codex-connector bot reviewed Feb 27, 2026

View reviewed changes

apps/app/src/app/api/security/vulnerability-reports/_api-proxy.ts Outdated Show resolved Hide resolved

apps/api/src/security-vulnerability-reports/security-vulnerability-reports.service.ts Outdated Show resolved Hide resolved

feat(security): add vulnerability reports flow with mock checkout and…

6a1e273

… Maced webhook reconciliation

vercel bot had a problem deploying to Preview – app February 28, 2026 01:21 Failure

github-advanced-security AI found potential problems Feb 28, 2026

View reviewed changes

apps/api/src/security-vulnerability-reports/security-vulnerability-reports.service.ts Fixed Show fixed Hide fixed

vercel bot deployed to Preview – portal February 28, 2026 01:23 View deployment

chatgpt-codex-connector bot reviewed Feb 28, 2026

View reviewed changes

apps/app/src/app/api/security/vulnerability-reports/_api-proxy.ts Outdated Show resolved Hide resolved

apps/api/src/security-penetration-tests/security-penetration-tests.service.ts Show resolved Hide resolved

fix(security): address review feedback for vulnerability reports

c26dfbf

vercel bot temporarily deployed to Preview – portal February 28, 2026 02:01 Inactive

vercel bot had a problem deploying to Preview – app February 28, 2026 02:02 Failure

claudfuen added 2 commits February 27, 2026 21:03

Merge remote-tracking branch 'origin/main' into codex/security-vuln-r…

e406e2d

…eports-module

fix(app): correct vulnerability report api proxy imports

fbaf81a

fix(security): harden pentest create flow and extract Maced client

be16674

vercel bot temporarily deployed to Preview – portal March 2, 2026 16:29 Inactive

vercel bot had a problem deploying to Preview – app March 2, 2026 16:30 Failure

chatgpt-codex-connector bot reviewed Mar 2, 2026

View reviewed changes

apps/api/src/security-vulnerability-reports/security-vulnerability-reports.service.ts Outdated Show resolved Hide resolved

feat(security): validate Maced client contracts and fix checkout URL …

789fb5e

…typing

vercel bot temporarily deployed to Preview – portal March 2, 2026 16:41 Inactive

vercel bot deployed to Preview – app March 2, 2026 16:43 View deployment

chatgpt-codex-connector bot reviewed Mar 2, 2026

View reviewed changes

apps/api/src/security-penetration-tests/maced-client.ts Show resolved Hide resolved

refactor(security): finalize penetration-tests naming and strict prov…

7273e28

…ider id contract

vercel bot temporarily deployed to Preview – portal March 2, 2026 17:10 Inactive

vercel bot deployed to Preview – app March 2, 2026 17:12 View deployment

chore(security): polish penetration-tests naming and config formatting

fa663be

vercel bot temporarily deployed to Preview – portal March 2, 2026 17:17 Inactive

chatgpt-codex-connector bot reviewed Mar 2, 2026

View reviewed changes

apps/api/src/security-penetration-tests/security-penetration-tests.service.ts Show resolved Hide resolved

apps/api/src/security-penetration-tests/security-penetration-tests.service.ts Outdated Show resolved Hide resolved

vercel bot deployed to Preview – app March 2, 2026 17:19 View deployment

fix(security): harden webhook handshake and reduce progress provider …

77ab81e

…calls

vercel bot temporarily deployed to Preview – portal March 2, 2026 18:48 Inactive

vercel bot temporarily deployed to Preview – app March 2, 2026 18:48 Inactive

chatgpt-codex-connector bot reviewed Mar 2, 2026

View reviewed changes

apps/api/src/security-penetration-tests/security-penetration-tests.service.ts Outdated Show resolved Hide resolved

feat(api): reconcile penetration webhook by runId and drop orgId query

4151251

vercel bot deployed to Preview – app March 2, 2026 19:53 View deployment

vercel bot deployed to Preview – portal March 2, 2026 19:54 View deployment

Merge branch 'main' into codex/security-vuln-reports-module

49f6031

vercel bot temporarily deployed to Preview – portal March 2, 2026 19:56 Inactive

vercel bot deployed to Preview – app March 2, 2026 19:58 View deployment

chatgpt-codex-connector bot reviewed Mar 2, 2026

View reviewed changes

packages/db/prisma/migrations/20260302113000_add_security_penetration_test_runs/migration.sql Outdated Show resolved Hide resolved

apps/api/src/security-penetration-tests/security-penetration-tests.service.ts Outdated Show resolved Hide resolved

fix(api): align webhook trust checks and run ownership migration

b91d508

vercel bot deployed to Preview – app March 2, 2026 20:09 View deployment

chatgpt-codex-connector bot reviewed Mar 2, 2026

View reviewed changes

		if (response.status < 200 \|\| response.status >= 300) {
		throw new Error(response.error ?? `Request failed with status ${response.status}`);

Conversation

claudfuen commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Final architecture

User flow

Canonical provider contract

What changed

Frontend (apps/app)

Backend (apps/api)

Database (packages/db)

Docs (packages/docs)

Environment variables

Required

Optional

Rollout

Out of scope (intentional)

Risk/cost note

Uh oh!

vercel bot commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Mar 2, 2026

Choose a reason for hiding this comment

claudfuen commented Feb 27, 2026 •

edited

Loading

Frontend (`apps/app`)

Backend (`apps/api`)

Database (`packages/db`)

Docs (`packages/docs`)

vercel bot commented Feb 27, 2026 •

edited

Loading